Skip to content

ci: simplify baseline benchmark job#21

Merged
mccutchen merged 24 commits intomainfrom
continuous-benchmarking-III
Jan 5, 2025
Merged

ci: simplify baseline benchmark job#21
mccutchen merged 24 commits intomainfrom
continuous-benchmarking-III

Conversation

@mccutchen
Copy link
Owner

@mccutchen mccutchen commented Jan 3, 2025 •
edited
Loading

This simplifies the baseline benchmark job in preparation for landing a pr job that will actually implement continuous benchmarking. End state we're aiming at:

  • bench / baseline runs on merges into main, reports results as part of the step summary, and caches results for later comparison to pull request benchmarks
  • bench / pr runs on each pull request and uses benchstat to compare cached results from the main branch to results from current commit on pull request and reports diff as a comment (TODO)

@codecov
Copy link

codecov bot commented Jan 3, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.39%. Comparing base (6b3cbf1) to head (67e85fd).
Report is 1 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main      #21   +/-   ##
=======================================
  Coverage   79.39%   79.39%           
=======================================
  Files           3        3           
  Lines         427      427           
=======================================
  Hits          339      339           
  Misses         64       64           
  Partials       24       24

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link

github-actions bot commented Jan 4, 2025

benchmarks: 96b55f4

go test -bench=. -benchmem -count=10
goos: linux
goarch: amd64
pkg: github.com/mccutchen/websocket
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkReadFrame/1KiB-4         	 1000000	      1018 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1018 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1020 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1017 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1021 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1019 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1035 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1075 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	 1000000	      1016 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1KiB-4         	  999866	      1029 ns/op	    1080 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1310	    868669 ns/op	 1048646 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1386	    859706 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1389	    850176 ns/op	 1048644 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1406	    851473 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1387	    853955 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1383	    860249 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1381	    853958 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1374	    849712 ns/op	 1048646 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1386	    855558 ns/op	 1048647 B/op	       5 allocs/op
BenchmarkReadFrame/1MiB-4         	    1393	    849977 ns/op	 1048648 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     183	   6388136 ns/op	 8388680 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     183	   6426241 ns/op	 8388678 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     184	   6371460 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     188	   6368170 ns/op	 8388678 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     184	   6388610 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     187	   6378449 ns/op	 8388679 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     187	   6348856 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     186	   6378997 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     188	   6393935 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/8MiB-4         	     187	   6413024 ns/op	 8388677 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      94	  13067256 ns/op	16777292 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      84	  13170204 ns/op	16777293 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      87	  13055939 ns/op	16777291 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      84	  13311856 ns/op	16777292 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      87	  13101520 ns/op	16777293 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      91	  13234573 ns/op	16777290 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      86	  13204986 ns/op	16777293 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      88	  13050826 ns/op	16777295 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      87	  13107562 ns/op	16777288 B/op	       5 allocs/op
BenchmarkReadFrame/16MiB-4        	      87	  13104817 ns/op	16777295 B/op	       5 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1228	    963107 ns/op	 1048677 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1255	    952874 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1238	    951589 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1231	    954956 ns/op	 1048677 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1250	    947693 ns/op	 1048677 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1244	    951625 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1249	    957110 ns/op	 1048677 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1245	    953783 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1242	    949834 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/1-4     	    1240	    952720 ns/op	 1048676 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     168	   7183896 ns/op	 8388710 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     166	   7182485 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     168	   7148418 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     164	   7102935 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     168	   7124646 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     165	   7153678 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     168	   7146265 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     165	   7119747 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     166	   7077343 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/8MiB/1-4     	     169	   7160729 ns/op	 8388709 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      84	  14502787 ns/op	16777320 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      85	  14813335 ns/op	16777320 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      81	  14563057 ns/op	16777321 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      81	  14520743 ns/op	16777321 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      85	  14400474 ns/op	16777321 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      81	  14452365 ns/op	16777321 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      85	  14547842 ns/op	16777319 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      85	  14400337 ns/op	16777320 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      85	  14779902 ns/op	16777320 B/op	       6 allocs/op
BenchmarkReadMessage/16MiB/1-4    	      82	  14460984 ns/op	16777321 B/op	       6 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     846	   1419895 ns/op	 3776819 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     805	   1402285 ns/op	 3776819 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     799	   1395335 ns/op	 3776817 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     866	   1362444 ns/op	 3776818 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     904	   1398148 ns/op	 3776818 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     739	   1429467 ns/op	 3776819 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     841	   1532872 ns/op	 3776819 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     782	   1353557 ns/op	 3776818 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     744	   1412809 ns/op	 3776820 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/4-4     	     844	   1441784 ns/op	 3776818 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     123	   9703085 ns/op	29958455 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     122	   9666075 ns/op	29958462 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     121	  10009064 ns/op	29958462 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     116	   9874921 ns/op	29958467 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     117	  10614916 ns/op	29958454 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     116	  10209930 ns/op	29958452 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     123	   9883781 ns/op	29958453 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     111	  10527609 ns/op	29958452 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     121	   9614889 ns/op	29958455 B/op	      24 allocs/op
BenchmarkReadMessage/8MiB/4-4     	     112	  10601316 ns/op	29958451 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      64	  18535421 ns/op	59859253 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      60	  18310731 ns/op	59859252 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      61	  18021015 ns/op	59859252 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      63	  18493761 ns/op	59859255 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      64	  19063917 ns/op	59859251 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      58	  18372484 ns/op	59859253 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      62	  18312415 ns/op	59859258 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      58	  18018639 ns/op	59859255 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      57	  18410021 ns/op	59859255 B/op	      24 allocs/op
BenchmarkReadMessage/16MiB/4-4    	      64	  19120037 ns/op	59859255 B/op	      24 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     546	   2075473 ns/op	 5669972 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     556	   2113251 ns/op	 5669975 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     556	   2197967 ns/op	 5669978 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     567	   2145571 ns/op	 5669974 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     574	   2058063 ns/op	 5669978 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     552	   2103536 ns/op	 5669974 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     584	   2030127 ns/op	 5669973 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     562	   2094501 ns/op	 5669976 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     578	   2158238 ns/op	 5669976 B/op	      90 allocs/op
BenchmarkReadMessage/1MiB/16-4    	     556	   2115243 ns/op	 5669978 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      63	  17354240 ns/op	49513568 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      84	  18454098 ns/op	49513573 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      81	  16225358 ns/op	49513572 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      70	  16147856 ns/op	49513560 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      76	  14787031 ns/op	49513562 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      68	  18248445 ns/op	49513570 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      66	  17607337 ns/op	49513574 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      66	  16253124 ns/op	49513569 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      60	  17456185 ns/op	49513562 B/op	      90 allocs/op
BenchmarkReadMessage/8MiB/16-4    	      78	  17564419 ns/op	49513569 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      51	  22692126 ns/op	98362451 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      46	  23242527 ns/op	98362449 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      51	  22326742 ns/op	98362455 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      51	  22769235 ns/op	98362457 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      42	  24248839 ns/op	98362451 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      48	  26600634 ns/op	98362466 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      44	  26755936 ns/op	98362475 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      39	  27114130 ns/op	98362461 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      54	  25710798 ns/op	98362460 B/op	      90 allocs/op
BenchmarkReadMessage/16MiB/16-4   	      43	  26717358 ns/op	98362467 B/op	      90 allocs/op
PASS
ok  	github.com/mccutchen/websocket	199.880s

benchstats: 52e6799...96b55f4

@mccutchen mccutchen enabled auto-merge (squash) January 5, 2025 13:50
@mccutchen mccutchen disabled auto-merge January 5, 2025 13:50
@mccutchen mccutchen changed the title ci: fix file juggling in bench job ci: simplify baseline benchmark job Jan 5, 2025
@mccutchen mccutchen merged commit 0b6bcf4 into main Jan 5, 2025
8 checks passed
@mccutchen mccutchen deleted the continuous-benchmarking-III branch January 5, 2025 13:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@mccutchen